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Abstract 


The research presented in this thesis aims to extend the capabilities of human 
interaction proofs in order to improve security in web applications and ser¬ 
vices. The research focuses on developing a more robust and efficient Com¬ 
pletely Automated Public Turing test to tell Computers and Human Apart 
(CAPTCHA) to increase the gap between human recognition and machine 
recognition. Two main novel approaches are presented, each one of them tar¬ 
geting a different area of human and machine recognition: a character recog¬ 
nition test, and an image recognition test. Along with the novel approaches, 
a categorisation for the available CAPTCHA methods is also introduced. 

The character recognition CAPTCHA is based on the creation of depth 
perception by using shadows to represent characters. The characters are cre¬ 
ated by the imaginary shadows produced by a light source, using as a basis the 
gestalt principle that human beings can perceive whole forms instead of just 
a collection of simple lines and curves. This approach was developed in two 
stages: firstly, two dimensional characters, and secondly three-dimensional 
character models. 

The image recognition CAPTCHA is based on the creation of cartoons 
out of faces. The faces used belong to people in the entertainment business, 
politicians, and sportsmen. The principal basis of this approach is that face 
perception is a cognitive process that humans perform easily and with a high 
rate of success. The process involves the use of face morphing techniques to 
distort the faces into cartoons, allowing the resulting image to be more robust 
against machine recognition. 

Exhaustive tests on both approaches using OCR software, SIFT image 
recognition, and face recognition software show an improvement in human 
recognition rate, whilst preventing robots break through the tests. 
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Chapter 1 


Introduction 


In the last two decades, since the commercialisation in the nineties of the Internet, the number 
of users has grown exponentially until reaching more than 2.2 billion people 11771 . This is 
the result of its popularisation and incorporation into virtually every aspect of modern human 
life from daily affairs such as education, web search or goods shopping to more professional 
oriented tasks. Advances in the protocols and the services have brought a wide variety of 
services. The most important one is the World Wide Web (WWW) that communicates via 
the Internet a series of resources such as interconnected documents, linked by hyperlinks and 
URLs. 

Since its creation, the Internet has no centralised governance in either policies for access 
and usage, technological implementation, or management, and it is maintained by each con¬ 
stituent network with its own standards. Due to this fact, security has become an important 
issue for the users, companies and services. One of the primary sources of abuse on the Inter¬ 
net is spam, that targets electronic messaging services by sending unsolicited bulk messages 
indiscriminately, especially advertising, among other actions such as instant messaging spam, 
web search engine spam, spam in blogs, in wikis, in ads, in forums and in social networks, 
mobile phone messaging spam, and file sharing network spam. It became a serious problem 
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when the internet was opened up to the general public in the mid-90s. The fact that people 
have quick and easy access to the internet network made this problem grow exponentially in 
the following years, reaching proportions of 85% and 90% of all the emails in the world lll22ll . 

Besides the huge expansion it has experienced, spam is also a serious problem because of 
the property rights and the consumed resources. First of all, spam is difficult to get rid off 
because property rights in several countries are difficult to enforce. Nowadays in Europe, there 
is a new legislation that tries to reduce the quantity of spam coming from the continent [!59ll . 
Secondly, if we talk about resources, spam consumes shared resources such as bandwidth or 
the load of the servers, or private resources such as money and time. Finally, another serious 
issue that derives from the existence of spam is that it has become a tool for malware authors 
and phishers to abuse the Internet. 

Malware or malicious software is the term used for a diverse kind of hostile, intrusive, or an¬ 
noying software that can be used to gather personal or private information, or to harm computer 
operations. The most common forms of malware are viruses, worms, trojan horses, spyware, 
adware, and other malicious programs If 1491 . On the other hand, phishing is a software used 
to acquire information such as usernames, passwords, and credit card details by disguising it¬ 
self as a trustworthy entity in an electronic communication or transaction 11591 with the aim 
of stealing money. Spam can be used by malware authors and phishing software through un¬ 
solicited commercial e-mails to spread harmful software with the objective of identity theft or 
even worse; fee fraud. These software programs take advantage of the victim’s inexperience 
with technology or attempt to call on human greed for money (see Figure [TTT1) . 

One of the most effective methods for reducing the amount of spam circulating on Internet 
and ensuring safety for users is the use of CAPTCHAs. A CAPTCHA is a program that pro¬ 
tects internet companies and human users against spam or bots through the generation of grad¬ 
ing tests that most humans can pass but current computers cannot lf20l .The term CAPTCHA 
stands for Completely Automated Turing Test to Tell Computers and Humans Apart and was 
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Figure 1.1: Flow diagram of a phishing and malware attack through spam. 

firstly coined in 2000 by Luis von Ahn, Manuel Blum, Nicholas Hopper and John Langford of 
Carnegie Mellon University [11881 . 

The primary application of CAPTCHA is to prevent malicious attacks to the systems by 
spammers. However, they also serve to protect vulnerable systems, such as Yahoo or Hotmail, 
against e-mail spam, automated posting to forums, blogs and wikis as a result of commercial 
interests or harassment. Another important function is bit rate limiting when excessive use of a 
service is observed. 

Nowadays, most of the methods to discriminate humans from computers are based on op¬ 
tical character or image recognition, or sound recognition. In a word-based CAPTCHA, the 
characters are distorted to make its recognition more difficult for the bots. Among the ba¬ 
sic distortions, it can use translation, rotation (clockwise or counterclockwise) and scaling, 
among others such as sight angle, lighting effects, context, and camouflage ll38Tl . A word-based 
CAPTCHA test consists on an image that contains distorted and noisy characters or words. To 
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Type the two words: 

Figure 1.2: Word-based CAPTCHA extracted from http://www.captcha.net. 

solve this test, the user has to type the characters presented in the image. Usually, the distor¬ 
tions applied to the image are complicated enough to prevent a robot to recognise the word 
while allowing humans to do so. An example of common CAPTCHA used in current web 
applications can be appreciated in Figure 11.21 

An image-based CAPTCHA contains primarily an image that the user has to recognise. 
Amongst these tests, the user can be asked to implement different ki nds of actions; solve a 
quiz, match symbols, recognise faces, etc. Usually, the images do not appear straightforwardly, 
instead they can contain warping, occlusion or lighting effects to avoid being recognised by 
machines. The last type is a sound-based CAPTCHA, which was implemented in the first place 
for those users that cannot solve visual CAPTCHAs due to an impairment. The test presents an 
audio file that contains words, letters, or numbers, mixed with background noise, that the user 
has to type correctly. 

Even thought there are many CAPTCHA methods available to prevent spam circulating, 
there are many researchers that have developed techniques to break through them [70] 11301 
131 1 16111 since it means a technological advance in machine learning. Additionally, companies 
have exploited the fact that users find the tests annoying to create commercial DeCAPTCHAs 
to break the CAPTCHA tests automatically, without the direct intervention of the users. Due 
to these facts and the greed of spammers, most of the current tests are becoming obsolete. 
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In this thesis, the major motivation was the creation of advanced software tools that enables 
separation of humans and machines in an automated environment and increases the gap of what 
humans can recognise and machines cannot. The targeted strategies have exploited extremely 
difficult tasks related to image understanding and human perception. These objectives were 
established in order to prevent all the security breaches produced by spam and other forms of 
attack, which are also caused by the inexperience of using computer technology by the majority 
of users. The primary contributions of this thesis are the development of two efficient and robust 
CAPTCHA approaches and a categorisation for the current CAPTCHA tests. 


1.1 Contributions 

For the Visual-word based CAPTCHA: 

• identification of the issues on the current word-based CAPTCHAs; 

• development of a new type of characters based on 3D objects with 3D boundaries delim¬ 
ited by shadows H15CHI ; 

• design of an efficient algorithm to optimise the distortions applied to the characters and 
ensure safety against possible external attacks to break the code H150H : 

• exhaustive experiments to test the efficiency of the approach and improve the human 
friendliness regarding the current approaches available H150H . 

For the Image-based CAPTCHA: 

• identification of the issues on the current image-based CAPTCHAs; 

• development of a database of faces of well known people and a second database with 
cartoons and animals to create a final image that is the result of the morphing between a 
selected image from each database H151II ; 
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• design of an efficient algorithm to optimise the morphing between images and ensure 
safety against possible external attacks to break the code [115111 ; 

• exhaustive experiments to test the robustness of the approach and improve the human 
friendliness regarding the current approaches available 1115111 . 

Finally, the categorisation gives a classification for every kind of test available and for future 
techniques since it goes from three general branches to a subclassification that can be enlarge 
if necessary. 


1.2 Overview of the thesis 

This thesis has been organised in a self-contained manner. The initial chapter presents the 
fundamental aspects of the addressed technology and the corresponding state of the art, the 
following three chapters present the techniques used to develop the approaches presented in 
the thesis. The subsequent two chapters present the proposed approaches, fully explaining 
the algorithms and the results obtained. The last chapter concludes the work, presenting the 
conclusions and considerations for future research. This thesis is organised as follows: 

Following the introductory chapter, Chapter 2 presents an overview of CAPTCHA methods, 
as well as a survey of the available CAPTCHA tests. Important evaluation concepts, such as 
efficiency and robustness, and human friendliness, are explained, as they will be important 
in the later chapters. Also, several well-known commercial and published CAPTCHAs are 
presented along with one of the contributions of the thesis; a categorisation of the CAPTCHAs. 

Chapter 3 summarises the basic concepts in digital image manipulation used to create visual 
CAPTCHA tests. Firstly, the digital image warping and morphing tools are presented, which 
are used to create the pertinent distortions for both approaches. Additionally, a 3D computer 
graphics study is introduced, since it will play a major role in the development of the new con- 


7 






cepts that differentiate the new CAPTCHA tests presented in this thesis with the ones currently 
available. 

In Chapter 4 the digital image recognition tools are presented. These tools are used to eval¬ 
uate the efficiency and robustness of the approaches created. For the OCR-based CAPTCHA, 
the SIFT tool is explained, since it will be used to evaluate the grade of machine recognition 
for characters. It also presents the state-of-the-art study in face recognition techniques, because 
different techniques will be used to measure the capacity of machines to recognise the distorted 
faces created by the image-based CAPTCHA. 

Human perception and recognition theories are the focus of Chapter 5. The main aim of this 
chapter is the evaluation of the human friendliness of the approaches presented in this thesis. 
Human perception theories are explained in the two sections that the chapter is divided. The 
first section focuses on Gestalt psychology, which defines a branch of psychology than explains 
how human beings perceive objects when they are incomplete, which is used to create the OCR- 
based CAPTCHA. The second section focuses on face perception and recognition with the aim 
of creating a good interactive image-based CAPTCHA. 

Chapter 6 introduces the first approach: the visual word-based CAPTCHA. The developed 
scheme introduces a new concept in the creation of a word-based CAPTCHA: the use of shad¬ 
ows to represent characters. Additionally, it presents both the experiments made to evaluate 
the efficiency and robustness, and the human friendliness and the results for these experiments, 
along with a complexity analysis of the test and a brief discussion of these results. 

Chapter 7 focuses on the second approach, the image-based CAPTCHA. This scheme is 
developed with the aim of creating a more interactive and secure test. It uses distorted faces 
of well known people from diverse cultural sectors, such as politics, sports, cinematographic 
industry, etc. Following the lead of the first approach, it also presents both the experiments 
made to evaluate the efficiency and robustness, and the human friendliness and the results for 
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these experiments, along with a complexity analysis of the test and a brief discussion of these 
results. 

The conclusions are summarised in Chapter 8. The list of author’s publications is given at 
the end of the thesis along with the references used. 
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Chapter 8 


Conclusions and Future Work 


8.1 Conclusions 

The aim of the research presented in this thesis was to increase the gap of what humans can 
recognise and machines cannot. Additionally, the creation of more robust and efficient novel 
methods was targeted. The main focus was centred on creating CAPTCHA tests using human 
psychology and universal common knowledge. The first step towards the developed methods 
was to analyse the current approaches and distinguish their weaknesses and possible ways to 
improve them. This includes a research of computer vision software that allows machines to 
break through the tests. 

The research on the current methods available uncovered the necessity of a classification to 
categorise the algorithms by the computer vision techniques used and by human aptitudes. For 
the classification, three main categories have been considered: OCR-based methods, Visual 
non OCR-Based methods and non Visual methods. These categories have been divided into 
subcategories for a more accurate classification. Along with the sub-categorisation, an exten¬ 
sive analysis of the available methods and their reliability was presented in the thesis, reaching 
the following conclusions: 
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OCR-Based methods were the first CAPTCHAs to emerge. They had a quick expansion to 
many different web applications as well as many prototypes. Along with the expansion, several 
different programs to break through them arose which provoked an increase in difficulty in the 
tests. Nowadays, most users find the annoying and time consuming. 

- Visual non OCR-Based methods emerged to explore diverse sides of HIP methods. At the be¬ 
ginning they focused on solving quizzes or matching problems but rapidly expanded to many 
other areas. Also, their reliability increased with time, going for easy to break to more secure 
that OCR-Based methods. Their diversity make them more human friendly and less time con¬ 
suming. 

-Non visual methods arose as an alternative to visual methods due to some visual impairments 
users may have. They weren’t as successful as the others due to their difficulty and language 
restrictions. 


The second step in this research was the development of two novel methods to prevent 
spam and malicious software to break through web applications and increase security when 
login in. The first method uses shadows to represent characters. The shadow boundaries were 
chosen to develop the fact that humans can easily recognise objects and characters only by the 
shadows but machines cannot. The distortions applied to the images are based upon geometric 
transformations that include affine and perspective transformations. The approach based on 2D 
shadow characters shows an improvement in efficiency and robustness over the actual CAP¬ 
TCHAs. The visual word-based CAPTCHA using 3D models is based upon lighting effects 
to create 3D shadows boundaries. The performance of this algorithm highlights that using 3D 
models yields better results in terms of efficiency and robustness. These tests are more difficult 
to solve for computer vision techniques but they still remain easy for humans. In this method, 
one of the challenges faced was that people visually impaired or with mental illness as dyslexia 
should be able to recognise the characters. However, it is also necessary to make the tests 
difficult enough for the machines not to break through them. 
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Humans can easily recognise cartoons or sketches from famous people, even if they are 
rotated or manipulated. A machine cannot recognise this type of image because it does the 
matching by pattern or feature extraction and the original one is very different. The second 
method uses distorted faces of world famous people to create a test to secure web applications. 
The main basis for the development of this method was the innate ability of human beings 
to recognise faces. The distortions applied to the face images are based upon a feature-based 
morphing process with multiple pairs of lines. The performance of this algorithm highlights 
two facts; firstly, using distorted faces as a test increments the efficiency and robustness of the 
previous approaches and secondly, it increments the difficulty for face recognition techniques 
to break through our system. 


8.2 Future Work 

In addition to the developed work, there are some challenges that have appeared while devel¬ 
oping the second approach. The main focus addresses the level of distortions applied to the 
faces. The main reason is that a high distortion factor can make the faces indistinguishable 
and a low rate can make it to easy for the face recognition system to break through the test. 
To measure the appropriate levels of morphing, different variables and factors were taken into 
account; cross-dissolve factor range, human recognition capabilities and the cartoon or animal 
used in the destination image. Another important factor to take into account was that people 
with prosopagnosia have more difficulties when recognising and distinguishing human faces, 
and even though there is nothing much that can be changed in this approach, the only alternative 
to help the human users with this problem is which kind of faces can be used. 

Practical realisations of methods presented in this thesis have enabled a high efficiency and 
robustness in the OCR-based CAPTCHA approach and the Image-based CAPTCHA approach. 
On the other hand, these realisations have also uncovered several interesting topics for future 
research, as well as some issues that have not been yet adequately resolved. These include: 


127 



-Since human and machine recognition depends on the diverse distortions applied, it is nec¬ 
essary an optimisation of the warping and morphing techniques by improving the algorithms 
and creating smoother transitions for the original image to the distorted one. New morphing 
techniques should also be taken into consideration. 

-Evaluation study of face recognition by human users depending on geographical locations. 
Knowing the cultural background and social knowledge is an important factor to increase the 
success rate by users. Also, it will be necessary to update the database depending on the latest 
celebrities or personalities that are famous at that moment. 

-Although the developed methods can prevent machines to successfully pass the current CAP- 
TCHAs, as the computer vision techniques research advances similarly the CAPTCHAs should 
improve. Therefore, the techniques applied and the human psychology used should be further 
studied. 
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